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ABSTRACT 



This paper presents research results from evaluations of the 
effectiveness of coalitions, and also presents a discussion of the identified 
characteristics of effective coalitions, a measurement tool developed to 
assess coalition effectiveness, and the application of this tool. 
Characteristics of successful coalitions and partnerships that have been 
identified are: (1) positive transfer of authority; (2) open and direct 

communication; (3) open decision making; (4) shared influence over decision 
making; and (5) participant involvement. The exploration of the relationships 
among objectives, characteristics, long term outcomes, and indicators of 
success had resulted in the development of a framework for evaluation. This 
framework was used to construct a survey instrument that scores the presence 
of the identified characteristics of coalition effectiveness that is to be 
completed by managers of the partner agencies . Scores are used to determine 
the strength of the coalition at the outset of an evaluation and throughout 
to monitor changes. The approach has been used in three evaluations, one of 
which is reported as a case study of the use of the instrument. The case 
coalition is a partnership of four agencies: a large metropolitan hospital, a 
small drop-in center with a voluntary board, a public health department, and 
a women's health center. Focus group discussions showed why the changes 
documented by the survey instrument occurred. Other factors related to the 
success of a coalition may be incorporated into the instrument. These include 
a common philosophical base, the role and accountability of the coordinator, 
an understanding of differences in partner contributions, control of project 
structure, and control of project space. (SLD) 
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Measuring Success in Coalitions 

by Mary Thelander and Beth Hoen 



Introduction 

In evaluating community based programs, we have been asked to determine the 
effectiveness of coalitions and partnerships. We assisted agency partnerships to identify 
the management characteristics that enable organizations, communities and program 
participants to work together in the planning and delivery of community services and to 
achieve their mutual goals. As well, agencies wanted to know the barriers to working 
together and how to overcome them, and the strengths of the partnership that were critical 
to continuing joint programming. 

The coalitions we have evaluated are community-based and have as a long-term goal to 
improve child and/or family health and functioning. The programs target people who are 
disenfranchised, for example because of poverty, housing, citizenship status, and are 
isolated by culture and language. The agencies are committed to program participant 
involvement in the coalitions and are primarily in the health and social services sectors. 
The coalitions vary in size: an eleven agency coalition, an eight agency coalition and a 
four agency coalition. The resources available to the coalitions to support the coalition 
itself and its programming also vary. 

In this paper, we present the results of our work with coalitions to measure their 
effectiveness. The identified characteristics, their definitions, a measurement tool and its 
application are outlined. As well we present additional factors contributing to successful 
partnerships that are emerging in our continuing work. 
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Development of Characteristics 



The characteristics were first developed in 1 994 with a coalition of agencies that wanted 
to understand the development and change in their partnership over time. Working with 
the managers from each of the partner agencies and the coalition coordinator, we 
identified and defined characteristics the group hypothesized were critical or essential to a 
coalition’s success. 

The characteristics or determinants of success the group identified are consistent with the 
management philosophy and values of community-based organizations. The 
characteristics are: 

Positive transfer of authority: the successful sharing of control and authority over 
management and programming of the project by all partners, thus establishing and 
maintaining the partnership. 

Open and direct communication: describes the attitude and behavior of members of the 
partnership in sharing information. Communication occurs “in the open” at meetings 
with full participation of the partnership and not secretly outside of meetings. 

Open decision-making: describes the way in which all members of the partnership 
participate in decisions. Decisions are not taken outside of the partnership and imposed. 

Shared influence over decision-making: means that all partners are equally respected 
and involved in decision-making. No partner has greater influence than another. 

Participant involvement: means that participants of the programs offered are involved 
in the decision-making of the partnership and are represented on the partnership 
committee. 

The evaluation framework on the next page is an example of the relationships we have 
explored between the objectives, characteristics, long term outcomes and indicators of 
success. The indicators are provided only as examples. They could be used as the basis 
for information collection in addition to the tool described below which measures the 
characteristics themselves. In some evaluations, we have added questions about the 
expected outcomes for the coalition — new networks, coalitions and programs. 
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Evaluation Framework: 
Effective Coalitions 
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Tool 



The survey instrument as 
shown in the sidebar was 
developed to score (seven point 
scale: 1 — rarely; 7 — usually) 
the presence of the 
characteristics. It is to be 
completed by managers of the 
partner agencies. Managers 
score the characteristics at 
different points in time. The 
initial scoring serves as the 
baseline against which change 
can be shown. 

We have used the scores to 
determine the strength of a 
coalition at the outset of an 
evaluation. Focus groups also 
help a coalition to understand 
these scores and to determine 
how they want to change their 
coalition. Repeat 
administrations and focus 
groups help coalitions to 
monitor the changes occurring. 



Partnership Survey 

The following six variables are characteristics of successful partnerships. Please rate 
how consistently your coalition or partnership has achieved these characteristics over 
the last six months: 



1 Positive transfer of authority is the successful sharing of control and authority over 
management and program of the project by all partners, thus establishing and 
maintaining the partnership. 

1 2 3 4 5 6 7 

Rarely Usually 

2 Open communication describes the attitude and behaviour of members of the 
partnership in sharing information. Communication occurs "in the open" at meetings 
with full participation of the partnership, and not secretly outside of meetings. 

1 2 3 4 5 6 7 

Rarely Usually 

3 .Open decision-making describes the way in which all members of the partnership 
participate in decisions. Decisions are not taken outside of the partnership and 
imposed. 



1 2 3 4 5 6 7 

Rarely Usually 



A.Shared influence over decisions means that all partners are equally respected and 
involved in decision-making. No partner has greater influence than another. 



1 2 3 4 5 6 7 

Rarely Usually 

SPartidpant involvement means that participants of the programs offered are involved 
in the decision-making of the partnership and are represented on the partnership 
committee. 



1 2 3 4 5 6 7 

Rarely Usually 



Mary Thelander & Beth Hoen, May 1997 
Please contact us if you plan to use this tool: 4 1 6-488-9687, 416-778-4290 or 
email: bhoen@web.net 
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Application 



We have used the characteristics of successful coalitions in three evaluations. We are 
reporting here on one as a case example to show the application and usefulness of the 
instrument, specifically how using it assisted the case coalition in working together. 

The case coalition is a partnership of four agencies; the agencies include a large 
metropolitan hospital, a small drop-in centre with a voluntary board, a public health 
department and a women’s health centre. The group came together to increase the health 
and well-being of babies bom to women at risk. 

Figure 1 shows the scores for two administrations of the survey (September 1995 and 
April 1 996) with the case coalition. 



Figure 1. 



Characteristics of Successful Coaltions: 
Mean Scores for Case Examole Coalition 




Scale of : Positive transfer Open Open decision Shared Participant 

1 to 7 of authority communication making influence over involvement 

decisions 
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All but one of the characteristics were scored higher at the second administration. Using 
a focus group we were able to determine why this improvement in scores occurred. 

The change in positive transfer of authority was a result of clarification of expectations 
about what authority could and could not be transferred. This clarity of understanding 
among the partners was helpful in reaching agreement about what partners could decide 
together and what individual partners decide. It was accepted that control exercised by 
some partners was necessary to meet the requirements of their own governing bodies 
and/or their funders. 

Likewise shared influence over decisions came to be seen as open but not equal. Some 
partners had more weight than others. As one member said, “I don’t see that as bad; 
some have more experience or knowledge than others”. Although decisions were taken 
by one or another partner, discussion took place about the decision and why it was taken. 
The characteristic of open communication , undertaken frequently, contributed to the 
perceived success in decision-making. 

Related to communication was participation. The active participation of all partners in 
the project also meant there was commitment and understanding of the project’s purpose 
and the partnership’s respective roles. All partners had “doers” at the front-line; 
consequently many program questions or problems were sorted out “on the spot” with 
clear knowledge about what was going on. 

Active participation meant active recruitment and involvement of participants in 
decisions about the program and its administration. The program provided an 
honorarium to program participants to enable them to be a part of the partnership and to 
be “economically valued” as were the other members who were employed by partner 
agencies. Leadership of participants was valued and participants stated their own sense 
of empowerment: “no one can fire us”. 
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Other Factors Contributing to Successful Coalitions 



We have identified other factors through our continuing work in evaluating coalitions. 
Over time we may revise our tool as we further examine these factors. 

Agreement on a common philosophical base contributes to the commitment of a 
partnership. Partners need to have enough in common or an overlap in their own 
organizational and sector mandates to support the involvement of their parent 
organizations. There must be concurrence about why the project is important, what it is 
achieving and why success is occurring. It is the clarity and agreement about the 
philosophy, program mandate and program logic model that enables partnerships to be 
comfortable with transfer of authority, communication and decision-making. 

The role and accountability of the coordinator are major contributors to the success of a 
partnership. A coordinator needs to be hired only to coordinate and lead the coalition. 
Conflicts arise when coordinators have direct service responsibilities. As well, 
experience has shown that coordinators who are responsive and flexible can facilitate 
program changes to meet the changing needs of participants. 

The coordinator’s accountability and reporting relationship to the project is critical in 
enabling the partnership to work. Her accountability needs to be to the project as a whole 
and not to one of the partners. As one coordinator said, “No (partner’s) voice is any 
stronger than any other (partner’s) voice.” 

In our experience, partners recognize there are differences between them and variations in 
the resources they bring to a partnership; however over time, partners learn that these 
differences in contribution do not need to interfere with the functioning of the 
partnership. For example, in the case presented, the hospital was controller of the 
funding and a large powerful partner. However, the hospital’s relationship with and 
attitude toward the coalition was different from its relationship with other hospital 
programs. The coalition project had more latitude and was less intertwined or linked with 
the hospital decision-making. The philosophy of the coalition program and its 
community orientation was respected as different but important to the hospital. 

Two factors — control of project structure and control of space — are also seen as essential 
to the autonomy of a coalition. Some coalitions believe that as long as the project is not 
seen as an extension of one agency either through staffing or space, it can remain distinct 
while making use of contributed space and staffing. 
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Marketing of the coalition and recognition by the community of policy-makers and 
funders contributes to the identity of the coalition and its stability. Active marketing to 
governments about what a coalition does and achieves is essential. A coalition has to 
have enough meaning that if it were to disband there would be a sense of loss. 

Creation of communication networks, both computer and personal, are also important to 
the success of coalitions. Open communication is enhanced by ease of access to partners 
without constraint by management level or structure. In one coalition, a computer 
network came to be used and respected as an effective means of communication in 
addition to meetings, telephone and faxed memos and reports. 



Conclusion 

Our experience with coalitions leads us to conclude that the characteristics of successful 
coalitions can be measured and are useful to coalitions in making management changes 
helpful to achieving their goals. 



Mary Thelander 
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